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§0. Introduction 

A (2+l)-dimensional topological quantum field theory (TQFT) determines, for each 
(7 > 0, a projective representation (p^, Vg) of the mapping class group Mg of a closed 
oriented surface of genus g. This paper is concerned with the SO (3) TQFT at an rth root 
of unity, r > 5 prime. Those TQFTs were first constructed mathematically in [T]. The 
problem we consider is this: what is the closure of Pg{Mg)7 

For g = 1 and the SU(2)-theory, Kontsevich observed that the image is finite. A proof 
of finiteness both for the SU(2)-theory and the SO(3)-theory may be found in [G]; see 
also [J], for an early calculation from which the finiteness can be deduced. For g >2, the 
image was shown to be infinite, and its closure therefore of positive dimension [F]. In this 
paper, we identify the representation for g = 1 and show that the image is either SL2(Fr) 
or PSL2(Fr) depending on whether r is congruent to 1 or —1 (mod 4). For g > 2, on the 
other hand, we show that the image is as large as possible — that is, it is dense in the group 
of projective unitary transformations on the representation space of Mg. 

We are working with the SO(3)-theory because the statements are cleaner than for 
the SU(2)-theory. Using the density result here and the tensor decomposition formulas 
in Theorem 1.5 [BHMV], the closed images of the SU(2)-theory can be identified. The 
restriction to a prime r is dictated by a result of Roberts [R] of which we make essential 
use: the SU(2)-theory representations of Mg are irreducible for r prime. 

The case r = 3 is trivial since dim Vg = 1 for all g. The case r = 5 was treated in joint 
work with Freedman [FLW2]. Our proof depended on the observation that a Dehn twist 
acts with distinct eigenvalues. For r = 5, this means that the Mg representations sat- 
isfy the "two-eigenvalue property." The compact Lie groups G admitting a representation 
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with respect to which a generating conjugacy class has only two eigenvalues can be classi- 
fied, and the few possibilities can be reduced to one by an examination of the branching 
rules for the restriction Mg_i C Mg and dimension computations (especially the Verlinde 
formula). For r > 7, we no longer have the two-eigenvalue property, and the number of 
possibilities for G grows rapidly with r. Mainly, therefore, we depend on the branching 
rules. A crucial point is to prove that the representations are tensor- indecomposable, i.e., 
not equivalent to a tensor product of representations of lower degrees; this is precisely why 
the SO (3) case is simpler than that of SU(2). The reason tensor-indecomposability is so 
important is that, coupled with irreducibility, it implies that the identity component of 
the closure of pg{Mg) is a simple group, and this greatly shortens the list of possibilities. 

The original motivation for this work was topological quantum computation in the sense 
of [FKLW], [FLWl], and [FLW2]. As in [FLW2], there are also applications to the distri- 
bution of values of 3- manifold invariants. As a simple example, we show that the set of 
the norms of the Witten-Reshitikhin-Turaev SO (3) invariants of all connected 3-manifolds 
at ^ = ie^ is dense in [0, oo) for primes r > 5. 

§1. The S0(3)-TQFT 

There are several constructions of the SU(2) and the SO (3) TQFTs in the literature (e.g., 
[BHMV][FK][RT][T]). The SU(2) TQFT was first constructed mathematically in [RT], and 
the S0(3) TQFT in [T]. We will follow Turaev's book [T], where the construction of a 
TQFT is reduced to the construction of a modular tensor category. 

27Ti 27rz(r+l) 

Fixing a prime r > 5 and setting A = ie~ = e ^ , note that A is a primitive 2r-th 
root of unity when r = 1 mod 4, and a primitive rth root of unity when r = — 1 mod 4. 
In [BHMV] to construct TQFT using the skein theory, the Kauffman variable A is either 
a primitive 4rth or a primitive 2rth root of unity. When r = 1 mod 4 by the SO{3) 
TQFT we mean the TQFT denoted by K in [BHMV] with the above choice of A. When 
r = — 1 mod 4, the same construction still gives rise to a TQFT although A is only 
a primitive rth root of unity, which is also denoted by Vr here, but the decomposition 
formula in Theorem 1.5 [BHMV] does not necessarily hold. Consequently we have to 
distinguish between the two cases r = 1 mod 4 and r = — 1 mod 4. 

The modular tensor categories associated to the TQFTs Vr are described in [T, Chapter 
XH]. In particular, [T, Theorem 9.2] discussed the unitarity of the TQFTs. For the above 
choice of A's, the ribbon categories of [T, Theorem 9.2] are not modular because the S- 
matrices as given in [T, Lemma 5.2] are singular. (The Kauffman variable ^ is a primitive 
4rth root of unity for only even r's.) But it can be shown that the even subcategories 
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(see [T, Section 7.5]) are indeed modular and unitary [T][FNWW]. The even subcategories 
correspond to the restriction of the representation categories to the odd dimensional (or 
integral spin) "halves" in the quantum group setting [FK] . 

A modular tensor category consists of a large amount of data. For our purpose here we 
will only specify the isomorphism classes of simple objects, called labels of the associated 
TQFT, the (S-matrix, and the T matrix. More information is contained in Lemma 2. 

We write the quantum integers [k]A = a'^-a-^ • '^^^ label set of the theory is 
L = {0, 2, 4, . . . , r — 3}. The quantum dimension of the label i is given by di = [i + 1], 
the subscript A in [k]A will be dropped from now on, and the global dimension of the 
modular tensor category is D — y^J^ieA'^ + -'■l^ ^ • '^^^ <5-matrix S = (sy) can be 

r 

read off from Lemma 5.2 [T] as Sij — [{i + + 1)]. The T = (tij) matrix is diagonal 
with diagonal entries the twists 6^, which are computed in [KL, Proposition 6, p. 43] as 

0. ^ ^i(i+2)_ 



the S, T matrices give rise to a projective matrix representation of SL2(Z) if we make the 
following assignments: s —>■ S = ■ S,t T~^, i.e., the SO{3) TQFT representation 
Pso(3) for SL2(Z) is: 



The T-matrix corresponds geometrically to a Dehn twist, so the negative twists 9^ are 
the eigenvalues for the image of any Dehn twist on a non-separating simple closed curve. 

Remark: It seems to be generally believed that the two theories Vr and V2r constructed 
in [BHMV] correspond to the S0(3) and the SU(2) Witten-Reshetikin-Turaev TQFTs. 
Actually the S-matrix of the theory is not the same as that in the Witten-Reshetikin- 
Turaev SU(2) theory [Wi][RT]: the {i,j)-th entry differs by a sign (-1)*+^'. But this 
discrepancy disappears on restriction to the even subcategories; this is the reason that 
the Witten-Reshetikhin-Turaev TQFTs are always unitary, but the V2r theories are not 
unitary in general. This subtle point is due to the Frobenius-Schur indicators for self-dual 
representations, and will be clarified in [FNWW]. 




Pso(3)(s) 



and 



PSO(3)(^) 



(^-^■(^■+2)5,,.)- 
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Lemma 1: If A is a primitive 2rth root of unity and r > 3 is odd, then there exists a 
TQFT V2 and a natural isomorphism of theories such that 



Moreover, the S0{3) -theory representations of the mapping class groups Mg are irreducible 
for all primes r > 5. 

Proof: The decomposition formula is Theorem 1.5 of [BHMV]. 

To prove the second part, first consider the case r = 1 mod 4. Suppose the S0(3)- 
theory representation of the mapping class groups for a closed surface is reducible for a 
prime r. Then by the tensor decomposition formula above the SU(2)-thcory representation 
would be reducible, too. But this contradicts the result of [R]. Therefore, the SO(3)-theory 
representations are also irreducible. For r = — 1 mod 4, the same argument will work if 
a similar decomposition formula holds. Without such a formula, the irreducibility of the 
SO{3) representations of Mg can be deduced by following Roberts's argument [R]. □ 

Lemma 2: Let dr,g,hi,...,hk denote the dimension of the vector space associated by the S0(3) 
theory at an rth root of unity to a compact oriented surface S of genus g with k boundary 
components labeled hi, . . . ,hk G 2Z. Then we have the following: 

1 ) dr,o,h = ^hO} where S is the Kronecker delta. 

2) dr,0,hi,h2 = Shih2- 

3) drfiMMM is 1 if and only if the hi satisfy the triangle inequality (possibly with equality) 
and hi + h2 + hs < 2(r — 2); otherwise it is 0. 

4) (Gluing formula) Suppose S is cut along a simple closed curve, and Hx denotes the 
resulting surface with the two new boundary components both labeled by x. Then 



Proof: Parts l)-4) are the basic data of the Vr theory (see also [KL]). Parts 5) and 6) are 



V^2r(S)=V2'(S)^V;(S). 




easy consequences of l)-4). The Verlinde formula is derived in [BHMV]. 



□ 



§2. Tensor products and decompositions 
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Next we prove some technical results which enable us to establish that certain represen- 
tations are tensor indecomposible. We say a complex representation F of a compact Lie 
group G is isotypic if it is of the form = W<SiC^ for some irreducible representation 
W of G. If W is one-dimensional, we say V is scalar. Two representations of G are 
conjugate if one is equivalent to the composition of the other with an automorphism of 
G. 

Lemma 3: Let — > Gi — > G2 — ^ G3 — ^ 6e a short exact sequence of compact Lie 
groups and p : G2 ^ GL{V) an irreducible representation. If the restriction of V to G\ 
is isotypic, then there exist central extensions Gi and G2 of Gi and G2 respectively and 
representations a and r of G2 such that 

(1) the extension Gi is a normal subgroup of G2, 

(2) the quotient G2/G1 is isomorphic to G3, 

(3) the restriction of a to Gi is irreducible, 

(4) the restriction of t to Gi is scalar, and 

(5) the tensor product a ® r is equivalent to the composition of p with the central quotient 
map G2 — > G2. 

Proof: By hypothesis, is isotypic and so can be written as W ® 'CJ' . The span of 

p{Gi) is 

p{Gi)C = End(W^) C End(W^ ® C^) = End(V), 

where End(VF) maps to End(VF (8) C'^) hy x i-^ x (® Idfc. The image p{G2) lies in the 
normalizer End(VF)End(C'^) of p(Gi)C. Thus, p can be regarded as a map 

G2 (End(W^)End(C'=))* = {GL{W) x GLfc(C))/C*. 

Let 

G2 = G2 X(GL(W)xGLfc(C))/C* {GL{W) X GLfe(C)), 

be a central extension of G2, Gi the pre-image of Gi in G2 with respect to the central 
quotient map tt : G2 — > G2, and p the puUback G2 (GL(VF) x GLfc(C)) of p. Let a and 
r denote the compositions of p with the projection maps GL(VF) x GLfc(C) GL{W) and 
GL{W) X GLfc(C) ^ GLfc(C) respectively. The diagram 

Gi <— Gi — > G2 

i i i p 

GL{W) = {GL{W) X C*)/C* ^ GL{W) x C* ^ GL{W) x GLfe(C) 
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shows that the restrictions of a and r to Gi are irreducible and scalar respectively, and 
(5) is immediate. □ 

In the other direction, we have: 

Lemma 4: Let G2 be a compact Lie group and Gi a closed normal subgroup. Let V and 
W be irreducible representations of G2 such that V\gi is irreducible and W\gi is scalar. 
Then V (S>W is irreducible. 

Proof: As V and W are irreducible, 

dimEndc^lV') = dim(F O V*)^^ = 1; dimEndc^lW^) = dim(H^ ® W*)^^ = 1. 

Let 

V ®V* =^Vi, W ®W* =^Wj 

i j 

denote decompositions into irreducible G2-i'epresentations, numbered so that Vi and Wi 
are trivial (and therefore the other Vi and Wj are non-trivial). Thus, 

EndG^{V^W) = {{V^V*)®{W®W*))^' = ^{Vi^Wjf' ^ C. 

id {i,j\Vi^W*} 

Now, VFjIgi is trivial for all j, so Vi = Wj implies that Viloi is trivial. However, 
1 = dimEndGi(^) = dim(F ® V*)^' = J^dimF^^S 

i 

SO this is possible only for i = 1. Then Wj = V^ implies j = 1, so 

dimEndcall^® W^) = 1. 

n 

Lemma 5: For r > 5, the tensor product of any two non-trivial irreducible representations 
o/SL2(Fj.) has an irreducible factor of degree > 

Proof: This can be deduced from the character table [Sp] p. 160, whose notation we follow. 
Let xi and X2 be non-trivial irreducible characters of SL2(Fr). If 

X1X2 = exj + /x^ +9, 

then 

I 1 if X2 = Xi, 
1 otherwise. 

If db e Ti is a square, then x^^{db) = —1- If X2 = Xi» comparing values at ^5, e + / < 1, 
which is absurd. Thus ^ = 0, and thus |xi('^6)X2('^6)| = e + /. As x(l) > ^^~T^\x{db)\ for 
all non-trivial X) we get a contradiction. □ 
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Lemma 6: Consider a short exact sequence of compact Lie groups 

O^Gi ^PSL2(F^) ^0, 

where r > 7. If H2 is the inverse image of a proper subgroup H C PSL2(Fj.), and V is a 
representation of H2, then some irreducible factor of In.d'^V has degree > 

Proof: Without loss of generality, we may assume that V is irreducible. Let K = ker Gi 
GL(Ind^^F). Replacing G2 and H2 hy G2/K and H2/K respectively if necessary, we may 
assume the restriction of Ind^^ V to Gi is faithful. Suppose that the restriction of some 
irreducible factor W of Ind^^F to Gi fails to be isotypic. By Clifford's theorem, W is a. 
direct sum of mutually conjugate isotypic representations Wj' of Gi; the stabilizer of Wi 
can be regarded as a subgroup A of PSL2(Fr), and 

dimW^ = A;dimW^i[PSL2(F^) : A]. 

By the well-known classification of subgroups of PSL2(Ff.), each proper subgroup has index 
> ^-=^, so dimVF > We may therefore assume that the center of Gi is in the center 
of G2- By a theorem of Eilenberg and Mac Lane [EM], the obstruction to finding a section 
of G2 PSL2(F^) lies in {PSL2 {¥ r), Z{Gi)). As PSL2(F^) is perfect with universal 
central extension SL2(Fr), G2 contains a subgroup F isomorphic to PSL2(Fj.) or SL2(F^) 
which maps onto PSL2(Fj.). As 

Res^JndgV = Ind^^nrRes^^ V, 

we can reduce to the case that G2 is PSL2(Fj.) or SL2(Fj.). If G2 = PSL2(Fj.) and H2 is the 
preimage of H2 C G2 in SL2(Fj.), then Ind^^F, regarded as a representation of SL2(F^) 
is the same as Ind^^'-'^'^^t^, where V is V regarded as a representation of H2. Therefore, 
without loss of generality, we may (and do) assume G = SL2(Fj.). 
If H1CH2C G, 

IndgjndgjFi = Indg^Fi, 

so without loss of generality we may assume H is a, maximal proper subgroup. Aside 
from the trivial representation, SL2(Fr) has two irreducible representations of dimension 
< with characters Xp- As y is irreducible, it is a subrepresentation of the regular 
representation of H, and it follows that Ind^F is a subrepresentation of the regular repre- 
sentation of SL2(Fr). In particular, [G : H] diml^ reduces to or 1 (mod ^^), the former 
if dim!/ > 1, and 

[G : H] dimF < 1^ + (dimxj)^ + (dimx^)^ = ^ < 
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It follows that \H\ > 2(r + 1). By the classification of maximal subgroups of SL2(F^), this 
means that H is a Borel subgroup S of G or that \H\ e {24,60,120}. The irreducible 
representations of B all have degree 1 or r — 1. If dimF = 1, the induced representation 
has degree r + 1, which does not satisfy the congruence condition. If dimF = r — 1, the 
induced representation has degree — 1, which does not satisfy the inequality condition. 
This leaves a short list of possible triples (r, dimV, \H\). For r > 11, all can be ruled 
out by the congruence condition or the inequality condition. The only triples not ruled 
out are (7, 1,48) and (11, 1, 120). In each case, the degree of the induced representation is 
congruent to 1 (mod ^^), so V must be trivial. By [At] pp. 3, 7, the induced representation 
in each case has an irreducible factor of degree r — 1. □ □ 

Lemma 7: Consider a short exact sequence of compact Lie groups 

^ Gi ^ ^ r ^ 0, 

where T is SL2(Fj.) or PSL2(Fr), r > 5. Suppose V and W are representations 0/G2. If 
V 0W has 

(1) all its irreducible factors of degree < ^-j^, 

(2) at least one irreducible factor of degree ^-=^ which is Gi-scalar. 

(3) exactly one irreducible factor of degree 1, 

then either V or W is one-dimensional. 

Proof: Suppose first that V and W are irreducible and V satisfies hypothesis (1). 
By Clifford's theorem, we can write V\gi and VF|gi as direct sums of mutually conjugate 
isotypic representations V-^ and WJ^ respectively. Let H2 D Gi denote the subgroup of G2 
stabilizing both and . Then Ind^^VT" ® is a G2-subrepresentation ofV ®W . 
By Lemma 6, it follows that = and = W^. (More generally, if any tensor 

product of G2-representations satisfies (1), all of the irreducible constituents of all of the 
tensor factors are Gi-isotypic.) 

By Lemma 3, replacing Gi and G2 by central extensions if necessary, we can write V as 
Vfj ® Vr and W as Wa- ® Wr with Va- and W^r Gi -irreducible and and WV Gi-scalar. As 
explained above, any irreducible consistituent U oiVa ® Wa is isotypic for Gi ; passing to 
central extensions of Gi and G2 if necessary, we write U asUa^Ur, so Ua®{Ur®Vr® Wr) 
is a G2-subrepresentation ofV^W. Every irreducible factor of Ur^Vr^ Wj- is Gi-scalar. 
By Lemma 5, unless at least two of Ur, and W^- have dimension 1, their tensor product 
has an irreducible factor of degree > which is Gi-scalar. If dimF,- = dimWV = 1, 
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then V\gi and W\gj_ are irreducible; if X is any one-dimensional representation of Gi, 
then X* (8) V\gi is irreducible, so dim(X* <S)V<S) W)'^'- < 1. 

Thus, ifV<SiW additionally satisfies hypothesis (2), either dimT^ > 1 or dimWV > 1. 
Without loss of generality, we assume the latter is true. Unless dimUr = dimV^ = 1, 

V <SiW contains a factor which is the tensor product of the Gi-irreducible Ua with a Gi- 
scalar irreducible of dimension > This is impossible by Lemma 4. The situation is 
therefore that (8) WV is Gi-scalar, G2-irreducible and of dimension and and every 
factor of Va- (8) W^- is Gi -irreducible. Applying (2) and Lemma 4 again, we see further that 
every G2-ii'reducible factor of Va- <8) has dimension 1, so W* decomposes entirely 
into 1-dimensional pieces over G2- If V is any other irreducible representation such that 

V <SiW satisfies (1), then is a twist of W* by a character so (g) F^* also decomposes 
as a sum of (dimF^)^ characters over G2- 

Now we return to the original problem. U V <SiW satisfies all three hypotheses, there 
exists irreducible factors Vq and Wq of V and W respectively satisfying hypotheses (1) 
and (2). Without loss of generality, we assume dimVor = 1- Then every irreducible factor 
of F is a twist of Wq^, so they all have the same dimension d. By hypothesis (3), there 
exist irreducible factors Vi and Wj of V and W respectively such that Vi Wj has a one- 
dimensional G2-submodule. Thus WJ must be a twist of Vi, and {V <SiWj)\Gi decomposes 
entirely into (P 1-dimensional pieces. By (3), d= 1. 

n 

§3. Case g=l 

Theorem 1. The projective representation of Mi = Map(/S'-'^ x S^) = SL2(Z) given by 

27ri 

the SO(3)-theory ai A = ie~^ is the same as the projective representation obtained by 
composing the (mod r) reduction map SL2(Z) SL2(Fr) with the odd factor of the Weil 
representation of SL2(F^). 

We want explicit matrices for one of the two ^-=^-dimensional irreducible representations 
of SL2(Fr). To find them, we briefly recall the theory of Weil representations over finite 
fields [Ge]. Let Hr denote the Heisenberg group of order r^. We regard Hr as a central 
extension 

The extension class defines a symplectic form (in this case, an area form) on the quotient. 
Any automorphism of Hj. stabilizes the center and acts on F^, respecting this symplectic 
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form. Regarding SL2(Fj.) as the group of symplectic linear transformations of F^, we claim 
that its action lifts to H^. To make this explicit, let x and y be elements of Hj. whose 
images in form a unimodular basis. Let z be the generator of the center defined by 



yxy X . Finally, for M 



a b 
c d 



e SL2(F^), let 



fMix) = z-^x-y^, fMiy) = ^'W, fM{z) = z. 

We easily check that this defines an action of SL2(F^) on H^. 

Let e(fc) denote e^^. Let (p, V^) denote the Stone-von Neumann representation (i.e., 
the unique irreducible representation of with central character ^ ^(/c).) We fix a 
basis Co, ... , e^-i for V so that is the e(2/c)-eigenspace of x. In this basis, we write 



p{x) = 



(\ 

e(2) 




\0 






e(4) 











<-2)/ 







1 


•• 












1 • • 


• 


, p{y) = 


















•• 


■ 1 


J 


\1 





•• 


• 0/ 



p(^) = e(l)Id. 



For any a e SL2(Fr) C Aut(iJr), poa is equivalent to p. There exists Ra-, therefore, unique 
up to scalar multiples, such that 



Rap{h)R-^ = p{a{h)) 



(*) 



for all h G Hr- If Ra denotes the class of R^ in PGL{V), we conclude that a ^ Ra 
is a projective representation. When r > 5, SL2(Fr) is perfect and centrally closed, so 
there is a unique lifting to an r-dimensional linear representation, which we call the Weil 
representation of SL2(Fj.). Explicitly we may choose (up to scalar multiplication) 

Rs = (e(2zj))o<ij<r, Rt = ((^ye(-Z^))o<i,j<r- 

We can verify (*) by checking it for h = x and h = y. 
Let = {0, 2, 4, . . . , r - 3} and set 



fi = e. 



Cr+l+i, i G E. 



It is easy to see that the span of /o, /2, /4, • • • , /r-3 forms an invariant subspace V°'^'^ of 
both Rs and Rt- In terms of this basis, Rs is represented by the matrix 



^odd 



1 



1 



1 



1 



ijEE 
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and Rt is represented by 

2^ 



r-l-j 



As 

A' = e 

we obtain 



2 fr + 1 



^odd ^ /^2(i+l)(i+2) _ ^-2(i+l)(i+l)^ ^ (^2 _ ^-2)([(^ + + 1)]^) 



and 

Thus the composition SL2(Z) ^ SL2(Fj.) PGL(F°'^'^) is equivalent to pso(3)- □ 



§4. Case g=2 

In what follows, we write for 



0<n<- 



Lemma 8: Let G he a simple compact Lie group with Coxeter number h, r > 5 a prime, 
and p : G ^ GL{V) an irreducible representation of G such that r divides dimF. If there 
exists g & G such that the spectrum of p{g) is X^, then r < 2h — 5. 

Proof: Let T be a maximal torus containing g and ( , ) denote the Cartan-Killing form on 
the character space X*{T) R. Let 

[a, a) 

and fix a Weyl chamber. If V has highest weight A and p is the half sum of positive roots, 
the Weyl dimension formula ([Bo] VIII, §9, Th. 2) asserts 

diml/=ni:^4±i^, 

a>0 ' 

where the product is taken over all positive roots. Let (3 denote the highest root. Since r 
divides dim V, and (//, a) e Z for all weights p, and roots a, 

{X,P) + {p,(3)>r, 
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or, by [Bo] VI, §1, Prop. 29(c) and [Bo] VI, §1, Prop. 31, 



(A, /?) + /i - 1 > r. 

The string of weights A, A -/?,... has length 1 + (A, (5) ([Bo] VIII, §7, Prop. 3(i)). For any 
w in the Weyl group, the string w(A), w{\)—w{f3)^ . . . has the same length. The Weyl orbit 
of P consists of all long roots ([Bo] VI, §1, Prop. 11), so the lattice it generates contains 
the root lattice if G is of type A, D, or E; twice the root lattice if G is of type B, C, or 
F; and three times the root lattice if G is of type G ([Bo] Planches). Since the difference 
between weights in an irreducible representation belongs to the root lattice, and since the 
eigenvalues of p{g) are rth roots of unity, not all equal, we conclude that if r > 5, w{l3){g) 
is a primitive rth root of unity for some w e W . A non-trivial geometric progression in 
Xr has length < so 

l + (A,/9)<^. 

Thus, r <2h-5. □ 

3 

Lemma 9: Under the hypotheses of Lemma 8, if^xmV = then G is a classical group. 

If V is not self -dual, then G is of type An. 

Proof: The Coxeter numbers of G2, -^4, -E'e, -E'7, Eg are 6, 12, 12, 18, 30, respectively 
([Bo] Planches). Examination of all primes < 55 [MP] reveals that the only case in which 

3 

^^^2^^ is the dimension of a representation of a suitable exceptional group is r = 7. The 
group is 6*2, and V is the adjoint representation. This case is excluded, however, as the 
longest string of short roots is 4 > Thus /3{g) = 1 for all short roots /5. As the short 
roots generate the root lattice, g has all eigenvalues equal, contrary to assumption. 

If V is not self-dual, G cannot be of type Bn or Cn and can only be of type Dn if n is 
odd and the highest weight A = aiAi -I — ■ -|- a„An satisfies sup(an_i, a^) > [MP]. As the 
Weyl dimension formula is monotonic in each Oj, dim V > 2"~^ while r < 2h — 5 = An — 9. 
Given the dimension of V, the only possibilities for (n, r) are (5, 11), (7, 13), (7, 17), (7, 19), 
(9, 19), (9,23), (11,31). For r < 31, the longest geometric progression in Xr has length 
< 4, so /i > r — 2 or r < 2n. This leaves only the case (7, 13), and by [MP], the only 
irreducible 91-dimensional representation of is self-dual. □ 

Lemma 10: For n > 11, if G — SU(n) and V is an irreducible representation of G with 
highest weight A and dimension < 2(3), then A is one of the following: 
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Highest weight 


Dimension 





1 


Al, An-1 


n 


A2, An-2 


(2) 


2Al, 2An-l 


("J') 


Al + An-l 




A3, An-3 


G) 


3Ai, 3An-i 





Proof: By the monotonicity of the Weyl dimension formula, it is enough to check that 
the dimension is always greater than 2(3) in the following cases (and their duals): Afc, 
4 < A; < n - 4; 4Ai; 2A2; 2A3; Ai + A2; Ai + A3; A2 + A3; Ai + A„_2; Ai + An-3; A2 + A„_2; 
^^2 + A„_3; A3 + A„_3; 2Ai + A„_i. □ 

Lemma 11: Let r >7 be a prime, G a simple compact Lie group, and p : G ^ GL{V) an 
irreducible representation of G such that 

1) V is not self dual, 

2) dimF=4=^, 

3) There exists g & G such that the spectrum of p{g) is X^. 

Then either G = SU(diml/), and V is the standard representation or its dual; orr = 13, 
G is a central quotient o/SU(14), and V is the exterior square representation or its dual. 

Proof: By Lemma 8, r < 2/i — 5. By Lemma 9, G is of type An, so n> so 

/n\ (r + 3)(r + l)(r- 1) r^ - r 
\s) ~ 24 ^ 24 ■ 

If n + 1 < 11, r < 17, so there are three cases: r = 7, n > 5, and dim]/ = 14; r = 11, 
n >7, and dimF = 55; and r = 13, n > 8, and dim]/ = 91. For n < 10, we see there is 
just one possibility for an irreducible representation of SU(?i + 1) of the given dimension: 
r = 11, n = 10, and V is the exterior square of the standard representation of SU(ll) or 
its dual. If n + 1 > 11, by Lemma 10, either V or V* has highest weight in the set 

{Al, 2Ai, 3Ai, A2, A3, Al + An-i} 

As V is not self-dual, we can exclude Ai + A^-i. For r > 7, 

(r + l)r(r-l) (r - l)(r - 2)(r - 3) 
24 ^ 8 ' 
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so C^) < only when m < r — 1, in which case equahty is ruled out since r does not 
divide C^). Applying this when m = n ± 1, we exclude the cases A3 and 3Ai. For A2 and 
2Ai, we seek solutions of 

(m\ 7-3 _ J, 
2y 24 

in integers m. For any such solution, r|m or r | m — 1, so 

12a(ar ± 1) = - 1, a e N. 

The discriminant of the quadratic equation for r is (12a^)^ + 4 ± 48a, and 

(12a^ - if < {12a^f + 4 - 48a < (12a^)^ < (12a^)^ + 4 + 48a < (12a^ + 1)^ 

for a > 3. For a = 2, the discriminant is not square for either choice of sign. For a = 1, 
we get the two solutions (r, m) = (11, 11) and (r, m) = (13, 14). An exhaustive analysis of 
sets S of rth roots of unity, whose symmetric or exterior squares give Xn or X13 reveals 
exactly two possibilities: the set 

{l,Cl3,Cl3,C?3} (1) 

and its complex conjugate have exterior square X13. □ 
Let Pg : Mg PGL(Vg) denote the projective unitary representation given by the 
SO(3)-theory. Let Gg denote the closure of the image. It is a subgroup of PSU(dimVg) 
and therefore a compact Lie group. We will often regard as a projective representation 

of Gg. 

Theorem 2. For (7 = 2, the projective representation p associated to the SO (3) theory at 
A = ie^ for r > 5 has dense image. 

The proof will be carried out in several steps. By [FLW2], we may assume r > 7. 
Step 1. The Lie group G2 is infinite. 

Proof: Consider the decomposition of the representation space arising from a curve sep- 
arating a genus 2 surface into two genus 1 surfaces with boundary. The components are 
indexed by labels 0, 2, . . . , r — 3, and they are projective representation spaces of Mi x Mi. 
The representation associated with label 21 is of the form W21 ® where each tensor 
factor has dimension ■ For label r — 5, it has dimension 2. Thus we have a two- 

dimensional projective unitary representation of Mi = SL2(Z); the ratio of eigenvalues for 
a Dehn twist is a primitive rth root of unity. By the classification of finite subgroups of 
SO (3), this implies the image is infinite, and it follows that the same is true for G2- 
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Step 2. The projective representation V2 is not self-dual. 

Proof: Equivalently, for any central extension M2 for which one can lift V2 to a linear 
representation (also denoted V2), the contragredient representation V2 is not obtained by 
tensoring V2 by a central character. We compute the multiplicities of the eigenvalues of a 
lift to M2 of a Dehn twist. These are just the dimensions dr,i,2i,2i of a doubly-punctured 
torus with both labels equal to 21, and are therefore given by 

(2Z+l)(r-2Z-l) 

dr,l,2l,2l - 2 • ^"^^ 

No two of these multiplicities coincide as I ranges over integers < so V2 cannot be 
self-dual. 

Step 3. Let G2 denote any central extension of G2 for which V2 lifts to a linear repre- 
sentation (which we also denote (p2, ^^2))- Let G2 denote the identity component of G2. 
Then the restriction of V2 to G2 is isotypic. 

Proof: As M2 is generated by Dehn twists, some lift i E G2 of a Dehn twist t would 
otherwise permute the isotypic components non-trivially. Thus, the eigenvalues of P2{i) 
(which are defined up to multiplication by a common scalar) would contain a coset of a 
non-trivial group of roots of unity [FLW2] Lemma 1.2. This is impossible, since up to 
scalars, the spectrum of a Dehn twist is Xr- 

Step 4. For any central extension G2 of G2 as above and any normal subgroup G2 such. 
that every homomorphism SL2(Fr) — > G2/G2 is trivial, V2 is tensor indecomposable as a 
(52-r epr esent ation . 

Proof: The restriction of V2 to 

Ml = {ifx^Mi C Mi^Mi C M2 

decomposes as a sum of terms of the form W2i. Let Hi denote the closure of Mi in GL(V2) 
and H'l the intersection of Hi with The occurrence of Wq as a factor guarantees 
that SL2(Fr) or PSL2(Fr) is a quotient of Hi for which the Wo-factor in question is an 
irreducible representation of degree the condition on G2 guarantees that H[ maps 
onto SL2(Fj.), so V2\fj, has an irreducible factor which is the composition of this quotient 
map and a degree representation of SL2(Fj.). Tensor indecomposability now follows 
from Lemma 7. 

Step 5. The restriction of V2 to G2 is irreducible. 
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Proof: Otherwise, by Lemma 3, there exists a central extension of G2 so that regarding 
V2 as a (52-representation, it has a tensor-decomposition. 

Step 6. The identity component is a simple compact Lie group. 
Proof: 

The identity component K = G2 is a connected compact Lie group. As V2 is an irre- 
ducible projective representation, the center of K is trivial. Therefore, it is a product of 
compact simple Lie groups Ki x ■ ■ ■ x Ks, and V2 is a tensor product of unitary projective 
representations Xi, . . .Xg of the Ki. In other words, 

KiX ■■■ X PSU(Xi) X • • • X PSU(X3) ^ PSU(V2), 

where the first inclusion is the product of inclusion Ki ^ PSU(Xj) and the second is 

3 

the tensor product map. As 2'^ > "^^-^ for all r, s < r. Consider the composition 
TT : G2 — Aut(K) — > Out{K). The outer automorphism group is contained in a product of 
groups of the form Out (Kj)** xS'g., where Sj < s. The largest proper subgroup of SL2(Fj.) is 
the Borel subgroup with index r + 1 > Si, so any homomorphism SL2(F^) — > is trivial. 
Therefore, any homomorphism from SL2(Fr) to Out(i^) lands in a product of solvable 
groups, and since SL2(F^) is perfect, that means any such homomorphism is trivial. By 
Step 4, V2 is tensor indecomposable as a representation of any central extension of the 
subgroup G2 = keiTT. However, g acts by inner automorphisms on K for g & G2, and since 
P2 is irreducible, this means G'2 C K C Y[i=i PSU(Xj), so s = 1. 
Step 7. The theorem holds if r 7^ 13. 

Proof: Applying Lemma 11 to the universal cover of G2, G2 is all of PSU(dim V2), so the 
same is true for G2- 

Step 8. The theorem holds if r = 13. 
Proof: 

For r = 13, we must consider the possibility that the universal covering group of G2 is 
SU(14), V2 is its alternating square, and a Dehn twist has exactly four different eigenvalues 
Xi in SU(14) given up to a common scalar multiple by (1) or its complex conjugate. By 
(3), the eigenvalues of a Dehn twist have multiplicities 6, 15, 20, 21, 18, 11 in PSU(dim V2), 
and each one arises uniquely as a product of distinct eigenvalues A^. Therefore, some 
must have multiplicity 11, but this is impossible since only one of the eigenvalues in V2 
has multiplicity divisible by 11. Therefore, G2 = PSU(dimV2) also for r = 13. 

□ 



16 



§5. Case g>3 



Theorem 3. For all r > 5 and all g >2, pg{Mg) is dense in PSU(dim Vg). 

For r = 5, this is already known [FLW2] Theorem 6.2. We therefore assume from now 
on that r > 7. We begin with a dimension estimate. 



Lemma 12: For r >7 and g >2, 



dimVi.,. < ) (2) 



except when r = 7 and g = 2. 
Proof: As cifc > 1 ^or all k, if g > 3, 



fc 

For 5f = 2, we compute 



dim^EoA ,r (r + 5)(r + 3)(r + l)r(r-l)(r-8) 

' dimV^a = 



2 J 5760 

and this quantity is obviously positive when r > 7. □ 

Proof of Theorem 3. 

The proof is very similar to that of Theorem 2. 
Step 1. The Lie group Gg is infinite. 

Proof: Consider the decomposition of the representation space arising from a curve sepa- 
rating a genus g surface into two pieces, one of genus 1 and one of genus g — 1. Restricting 
to Mg_i X Ml, we obtain a decomposition 

r-3 
2 



^5 = 0^.-1,2/ ®W^2/, 



where denotes the projective representation space of Mg_i associated to a surface 

of genus g — 1 with a single boundary component labeled 21. Now we proceed as in Step 
1 of Theorem 2. 

Step 2. The projective representation Vg is not self-dual. 
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Proof: If Vg is self-dual, its restriction to Mg_i x Mi decomposes into self-dual projective 
representations and mutually dual pairs of projective representations. We use induction on 
g, the base case being Step 2 of Theorem 2. By the induction hypothesis, Xg_ifi (8) Wq = 
Vg-i (g) Wo is not self-dual. Neither can it be dual to any other factor since Wq = Vi is 
irreducible and the other representations W21 have lower dimension. 

Step 3. Let Gg denote any central extension of Gg for which Vg lifts to a linear represen- 
tation. Let G°g denote the identity component of Gg. Then the restriction of Vg to G° is 
isotypic. 

Proof: Identical to the proof of Step 3 of Theorem 2. 

Step 4. For any central extension Gg of Gg as above and any normal subgroup, Vg is 
tensor indecomposable as a G^-representation. 

Proof: The restriction of Vg to 

Ml = {1} X Ml C Mg_7x Ml C Mg 

decomposes as a sum of terms of the form W21 ■ Now we proceed as in Step 4 of Theorem 
2. 

Step 5. The restriction of Vg to G° is irreducible. 

Proof: Identical to the proof of Step 5 of Theorem 2. 

Step 6. The identity component G° is a simple compact Lie group. 

Proof: Let K = G°. As in Step 6 of Theorem 2, is a product of compact simple Lie 
groups Ki, and Vg is the tensor product of unitary projective representations Xi of the 
Ki. Thus, 

KiX ■■■ X PSU(Xi) X • • • X PSU(X,) ^ PSU(Vg). 

The conjugation action of G2 on K must act transitively on the factors, since any decompo- 
sition into orbits gives a tensor decomposition of V. Therefore, the Ki are mutually isomor- 
phic, and their representations Xi are equivalent up to composition with automorphisms 
of Xi] in particular, their degrees are all the same. Now, the closure of Mg_i C Mg_i x Mi 
in K maps onto Gg-i since Vg-i = Xg_i is a summand of the restriction of Vg to Mg_i. 
Thus some factor Ki maps onto Gg-i. By the induction hypothesis, 

dim. Ki > dimGg_i = dim V?_i — 1, 
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so any non-trivial representation of Ki has degree > dimV^_i. Therefore, dirnVg > 
dimT^*_;^. The inequahty (2) then imphes s = 1 except possibly when g = 3 and r = 7, in 
which case, dimV^ = 98 < 14^ = dim.Vg_i, so again s = 1. 

Step 7. For all > 3 and r>7,Gg = PSU(dimVg). 

Proof: We use induction, the base case being Theorem 2. By the induction hypothesis 
and (2), 

rank > rank Gg_i > dim Vg_i - 1 > ^2dimVg + 1/4 - 1 /2. (4) 

By Step 2, Vg is not self-dual, so by [MP], Gg is of type An, Dn {n odd), or Eq. The 
case Eg is ruled out since dimV^_i > 7 in all cases ^ > 3. The minimal dimension for a 
representation of which is not self-dual is 2'^~^ > ("J^) for n > 4. This leaves the case 
where the inequality dimVg < ("2^) implies dimVg < 2 ("3^) for n > 2. Lemma 10 
gives the list of possibilities. For n > 7, the possible highest weights are Ai, Aj^, A2, Xn—i, 
2Ai, 2An, and Ai -|- A^, the last being ruled out as Vg is not self-dual. Up to duality, then, 
Vg is either the standard representation, its exterior square, or its symmetric square, and 
both are ruled out by (4). 

§6. An application 

Besides determining the representations of the mapping class groups, a TQFT also de- 
termines invariants of oriented closed 3-manifolds [RT]. To describe the S0(3) invariant of 
3-manifolds, we introduce the following notations: let di and 9i be the quantum dimension 
and the twist of the label i, and D the global dimension of the modular tensor category de- 
fined in Section 1; then we define p± — ^jg^, ^f^^l, and uq to be the formal sum ^^^^ 
If a 3-manifold is represented by a framed link L in ^S"^, then the 5*0 (3) 3-manifold 
invariant of is t(M^) = ■ {uJo*L) ■ where cr{L) is the signature of the linking 

matrix of L, and {u!o*L) is the link invariant of L where each component of L is labeled 
by ujo [T]. Note that in this normalization, t{S^ x S"^) = 1, t{S'^) = j^. The invariant r is 
multiplicative for disjoint unions, but for connected sums r(Mi7^M2) = D ■ t(Mi)t(M2). 

Recall that a TQFT 3-manifold invariant is only defined for extended oriented closed 
3-manifolds, but as pointed out in [A] there is a preferred framing for each oriented closed 
3-manifold; therefore, the formula above should be thought as for an extended 3-manifold 
with the preferred framing determined by the framed link L. The same 3-manifold invariant 
can also be defined using the representations of the mapping class groups. The subtlety in 
framing is reflected in the fact that the TQFT representations of the mapping class groups 
are only projective representations. It is known that ^ is a root of unity of finite order 
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[BK], so we can write ^ = for a rational number c, which is called the central charge 
of the TQFT (well-defined modulo 8). Framing changes lead to powers of k = e^. So 
up to powers of k, the same SO (3) invariant of 3-manifolds can be obtained as follows: 
suppose an oriented 3-manifold is given by gluing together two genus=^ oriented 
handlebodies Hg by a self-diffeomorphism / of = dHg (note that the handlebody Hg 
determines a vector vq in the TQFT vector space VriT^g) of up to a power of k); then 
the 50(3) 3-manifold invariant t(M^) is, up to a power of k, the inner product of vq with 
Pso{3){f){'Vo)- Theorem 3 has a direct corollary concerning the Witten-Reshitikhin-Turaev 
SO (3) invariants of 3-manifolds. 

Theorem 4. The set of the norms of the S0(3) invariants at A — ie^ of all connected 
3-manifolds is dense in [0, cxo) for primes r > 5. 

Proof: Given any complex number z, note that D > 1 so we can find a (7 > 1 so that 
z' = j-,g-i satisfies \z'\ < 1. Then arrange z' as the (1,1) entry of a unitary matrix Uz, and 
the vector vq determined by Hg as the first basis vector of an orthonormal basis of Vr{Ti). 
By Theorem 3, Uz can be approximated by a sequences of unitary matrices associated to 
diffeomorphisms fi of "Eg up to a phase. Each fi determines a 3-manifold Mf- by gluing 
two copies of Hg. It follows that the 3-manifold invariant of Mj- is the (1,1) -entry of 
Pso{3){fi) up to a phase, hence approximates z' up to a phase. By connecting sums Mf- 
with (g — 1) copies of x S'^, we approximate z using the invariants of the 3-manifolds 
Mf,i^{g-l){S^xS'). □ 

Remarks: (1): A similar density result of the SU(2) invariants for r = 1 mod 4 can be 
deduced from the SO (3) case and the decomposition formula. 

(2): This result does not follow from Funar's result [F]. The infinite mapping classes are 
in the image of the braid groups, and can be extended to the handlebody groups of Hg. 
Therefore, all resulting 3-manifolds have the same invariant up to powers of k. 

Finally, we make the following: 

Conjecture: The set of the SO (3) invariants at A = ie~^ of all connected 3-manifolds 
is dense in the complex plane for primes r > 5. 
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